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Purpose 


The  purpose  of  this  document  is  to  provide  an  overall 
plan  for  Verification,  Validation  and  Accreditation  (W&A)  of 
the  Aggregate  Level  Simulation  Protocol  (ALSP)  Confederation 
of  Models.  The  objectives  of  this  plan  are  to: 

1.  Increase  confidence  in  Confederation  of  Models  W&A  by: 

a.  Establishing  a  W&A  methodology  that,  when  applied, 
will  provide  feedback  to  model  developers  and  proponents 
regarding  model  and  interface  deficiencies;  provide  a  measure 
of  confidence  in  the  capability  of  1994  ALSP  Confederation  to 
meet  exercise  requirements;  and  provide  users  with  a 
characterization  of  model  capabilities  and  a  risk  assessment. 

b.  Develop  a  uniform,  rigorous  test  standard  for 
integration  and  operational  testing  of  the  ALSP  Confederation. 

c.  Provide  a  central  document  for  the  coordination  of 
activities  of  all  Confederation  testing  activities  to  ensure 
that  all  W&A  related  issues  are  addressed  in  a  complete  and 
timely  manner . 

2.  Increase  our  understanding  of  how  to  perform  W&A  of 
confederated  simulations . 

3 .  Collect  lessons  learned  for  future  W&A  activities  in 
support  of  confederated  simulations. 
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1 . 1  Background 

This  section  provides  a  background  discussion  of  the  1994 
ALSP  Confederation,  the  proposed  ALSP  Confederation  W&A 
process,  and  the  1994  ALSP  Confederation  W&A  Master  Plan 
(CWAMP)  .  The  discussion  begins  with  modeling  and  simulation 
(M&S)  including  activities  underway  to  connect  existing 
simulations  and  simulators,  the  simulations  currently  seeking 
to  participate  in  the  ALSP  Confederation,  and  the  history  of 
ALSP  to  the  present  1994  ALSP  Confederation. 

1.1.1  M&S  Background 

Effective  planning  for  the  1994  ALSP  Confederation  and 
for  future  joint  training  efforts  is  founded  in  knowledge  of 
existing  M&S  systems  and  their  historical  development. 

Simulations  are  generally  intensive  consumers  of 
computing  and  software  development  resources,  and  they  tend, 
therefore,  to  be  narrowly  tailored  to  give  the  best  possible 
results  for  their  proponents,  rather  than  addressing  general 
problem  areas . 

New  opportunities  are  now  emerging  to  integrate  the 
simulation  activities  developed  throughout  the  Department  of 
Defense  (DoD) .  In  response  to  this  opportunity,  the  Deputy 
Secretary  of  Defense  has  instituted  the  Defense  Modeling  and 
Simulation  Initiative.  The  goal  of  this  initiative  to  create 
an  M&S  based  virtual  battlefield  environment  in  which  to 
train,  plan  operations,  and  develop  and  test  doctrine.  The 
initiative  will  not  only  integrate  DoD  simulation  activities, 
but  will  also  spawn  new  applications  made  possible  by  the  new 
technologies  developed. 

1.1.2  M&S  Applications  and  Architectures 

For  ALSP  Confederation,  the  most  relevant  M&S  systems  are 
training  oriented  battle  simulation  systems .  A  number  of  M&S 
systems  have  been  developed  to  support  training  exercises  at 
various  echelons .  An  example  of  a  training  exercise  is  the 
command  post  exercise  (CPX) .  CPXs  provide  training  for  high 
level  command  staff  in  a  realistic,  interactive  environment. 
This  type  of  training  was  developed  to  remedy  perceived 
deficiencies  in  the  traditional  methods  of  training  these 
commanders .  CPXs  provide  a  realistic  combat  command  training 
experience  without  the  uncertainties  and  costs  of  actual 
warfare  or  field  training  exercises . 

A  CPX  places  the  command  and  control  (C2)  elements  of 
selected  units  in  a  combat  environment  that  stimulates 
decision  making  and  interaction  among  the  command  staff,  and 
develops  group  cohesion  and  coordination.  In  a  CPX,  the 
participating  officers  and  staffs  are  located  in  actual 
command  posts  or  tactical  operations  centers  at  field 
locations.  Controllers  play  roles  as  the  forces  above,  below, 
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and  adjacent  to  these  headquarters  (HQs)  ,  and  conimunicate  with 
them  through  unit  organic  communications .  The  controllers 
function  as  actors,  providing  a  realistic  interface  between 
the  training  audience  and  the  simulated  battle,  feeding  the 
trainee's  orders  into  the  model,  and  returning  battle  outcomes 
via  normal  command,  control  and  communications  (C3)  channels. 
Controllers  also  'play'  the  model  to  attain  desired  training 
effects . 

There  are  many  training  oriented  simulation  and 
stimulator  systems  extant,  each  tailored  to  the  training  needs 
of  a  particular  audience.  The  general  success  of  these 
systems  has  led  to  a  demand  to  broaden  the  scope  of  training, 
and  for  exercises  that  provide  Joint  Force  training. 
Integrating  existing  simulations  in  an  ALSP  Confederation  is 
an  attempt  to  provide  this. 

Because  of  the  high  availability  requirement  and  the 
large  resource  investment  required  to  carry  out  a  joint 
exercises,  an  effective  W&A  effort  applied  to  the  ALSP 
Confederation  will  ensure  a  valid  automated  exercise  support 
system  and  reduce  the  risk  of  failure. 

1.1.3  Existing  M&Ss 

Simulations  are  the  building  blocks  of  confederated 
training  simulations.  A  simulation  represents  all  simulated 
entities,  situations,  and  their  interactions  in  software. 
ALSP  provides  a  means  for  disparate  simulations  to  exchange 
information  and  function  as  a  confederation,  hence  the  ALSP 
Confederation. 

Some  examples  of  existing  simulations  are: 

a.  Corps  Battle  Simulation  (CBS) .  CBS  is  the  Corps  and 
Division  level  command  staff  trainer  in  the  Army's  Family  of 
Simulations  (FAMSIM)  .  It  is  used  as  a  CPX  driver  by  the 
Battle  Command  Training  Program  (BCTP)  and  by  the  individual 
corps  to  train  corps,  division,  and  brigade  staffs.  CBS  is 
also  used  by  BCTP  as  a  seminar  trainer.  It  is  currently  used 
more  than  100  times  per  year  by  I  Corps,  III  Corps,  V  Corps, 
VII  Corps,  XVIII  Corps,  National  Simulation  Center  (NSC) , 
BCTP,  United  States  (US)  Army  (USA)  Central  Command,  and  USA 
Command  and  General  Staff  College. 

b.  Air  Warfare  Simulation  (AWSIM) .  AWSIM  is  designed  to 
help  train  senior  North  Atlantic  Treaty  Organization  (NATO) 
commanders  and  their  battle  staffs  in  the  execution  of  wartime 
general  defense  plans  emphasizing  joint  and  combined 
operations .  The  model  is  used  to  develop  team  skills  and  to 
drive  unscripted  CPXs .  AWSIM  is  used  by  all  NATO  military 
commands  up  to  12  times  a  year,  depending  on  the  Warrior 
Preparation  Center's  (WPC)  exercise  schedule. 
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c.  Joint  Theater  Level  Simulation  (JTLS) .  JTLS  is  a 
simulation  designed  to  evaluate  theater  level  operations 
plans.  It  is  used  to  evaluate  different  mixes  of  forces  and 
logistics  support,  and  also  for  high  resolution  seminars. 

d.  Research,  Evaluation,  and  Systems  Analysis  (RESA) . 
RESA  is  a  research  and  evaluation  tool  for  Naval  C3  systems. 
It  is  used  for  operations  plan  evaluation,  C2  training  support 
for  senior  officers,  joint  C3  interoperability  assessment, 
warfare  systems  architecture  analysis,  and  naval  exercise 
support . 

e.  Combat  Support  Service  (CSS)  Training  Simulation 
System  (CSSTSS) .  CSSTSS  is  an  exercise  driver  for  training  of 
CSS  commanders  and  staff  personnel  in  C3 .  The  training 
audience  includes  the  CSS  commanders  and  staffs  in  Echelons 
Above  Corps,  Corps  Support  Commands;  and  Division  Support 
Commands  as  well  as  their  subordinate  HQs .  CSSTSS  is  used  to 
conduct  logistical  exercises. 

f.  Tactical  Simulator  (TACSIM) .  TACSIM  is  an 
interactive  simulation  supporting  Intelligence  and  Electronic 
Warfare  (lEW)  system  development  and  testing,  CPXs,  and 
testing  of  lEW  and  C3  functions.  It  supports  decision  making, 
systems  evaluation,  training,  and  the  All-Source  Analysis 
System  and  Enemy  Situation  Correlation  Element  program 
development . 

These  models  are  only  a  small  fraction  of  publicized  DoD 
simulations.  The  Joint  Chiefs  of  Staff  J-8  12th'  edition  of 
the  Catalog  of  Wargaming  and  Simulation  Models  lists  528 
simulations .  The  total  number  of  simulations  in  the  M&S 
community  is  much  larger. 

1.1.4  ALSP 

ALSP  is  both  a  software  system  and  a  data  communication 
protocol  that  allows  integration  of  disparate  simulations 
called  actors.  Under  ALSP,  different  simulations  can  join 
together  and  form  a  Confederation  of  Models.  This  becomes,  in 
effect,  a  composite  simulation.  Figure  1-1  shows  a  diagram  of 
single  ALSP  Confederation; 
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An  ALSP  confederation  has  two  main  parts : 


The  ALSP  software  system  -  the  ALSP  software  system  is 
composed  of  the  ALSP  Common  Module  (ACM)  and  the  ALSP 
Broadcast  Emulator  (ABE) .  The  ACM  serves  as  a  common  point  of 
interface  for  all  component  Actor  translators.  One  version  of 
the  ACM  will  function  for  all  Actors,  a  separate  copy  is 
provided  for  each  Actor  in  the  confederation.  The  ABE  serves 
as  a  message  router  for  the  confederation.  Depending  on  the 
distribution  of  the  simulations,  multiple  ABEs  may  be 
required. 

The  component  simulations  (Actors)  -  an  Actor  is  usually 
an  actual  warfare  simulation.  Each  simulation  wishing  to  join 
the  confederation  must  develop  a  translator.  Actors  differ 
and  the  translator  is  tailored  to  provide  a  bridge  between  the 
Actor  and  the  ALSP  Confederation. 

The  1993  ALSP  Confederation  had  the  following  members: 
CBS,  AWSIM,  RESA,  and  the  TACSIM  ALSP  translator.  Candidates 
for  the  1994  ALSP  Confederation  are  CBS,  AWSIM,  RESA,  TACSIM, 
Marine  Air  Ground  Task  Force  Tactical  Warfare  Simulation 
(MTWS) ,  Joint  Electronic  Combat  Electronic  Warfare  Simulation 
(JECEWSI),  and  CSSTSS. 

1.1.5  Confederation  W&A  (CWA)  History  and  Lessons  Learned 

A  brief  history  of  the  ALSP  Confederation  and  lessons 
learned  from  its  first  use  in  a  major  exercise  is  presented 
below  in  order  to  foster  a  better  understanding  for  the 
reasons  underlying  the  need  for  a  comprehensive  plan  for  W&A 
of  the  Confederation.  For  a  more  detailed  and  complete 
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history  or  the  ALSP  Confederation  and  its  past  uses,  the 
reader  is  referred  to  The  ALSP  Program  Status  and  History^ . 

Significant  advances  are  currently  being  made  in  linking 
together  simulations.  The  first  use  of  ALSP  linked 
simulations  to  support  a  training  exercise  was  Central 
Fortress  92.  The  linked  simulations  were  CBS  and  AWSIM. 
Support  for  Ulchi  Focus  Lens  (UFL)  92  and  REFORGER  92  was  also 
provided  by  these  ALSP  linked  simulations.  For  UFL  92,  the 
Navy  RESA  model  was  also  included  in  the  ALSP  linking.  The 
1992  joint  exercises  involved  thousands  of  participants 
distributed  over  several  continents.  In  1993,  the  joint 
community  saw  the  first  major  joint  exercise  supported  by 
multiple  simulations  linked  over  a  network  when  the  ALSP 
Confederation  successfully  supported  UFL  93 . 

While  the  overall  objectives  of  the  joint  exercises  were 
met,  a  concern  about  using  untested  research  oriented  support 
systems  surfaced.  The  need  for  a  confederation  testing 
process  to  ensure  reliability  was  realized  and  validity  of  the 
interactions  between  the  models  (i.e.  the  Confederation) 
significantly  impact  the  validity  of  the  joint  training 
exercise. 

Some  problems  originate  in  the  inherent  difficulties  of 
confederated  software  development  by  multiple  agencies. 
Others  result  from  the  requirement  that  software  designed  to 
run  stand  alone  be  retrofitted  to  work  in  cooperation  with 
other  systems.  The  need  to  establish  cooperative,  coordinated 
development  efforts  focused  on  Confederation  participation 
became  evident .  Communication  between  model  developers  is  key 
to  the  success  of  the  endeavor. 

Many  problems  associated  with  ALSP  confederation  testing 
resulted  from  the  different  configuration  control  and 
development  schedules  of  the  organizations  responsible  for  the 
testing.  These  problems  were  aggravated  by  the  lack  of  any 
single  point  of  authority  to  which  problems  could  be  addressed 
for  resolution.  Even  though  an  Interface  Control  Document 
(ICD)  existed  for  CBS  and  AWSIM,  there  were  inconsistencies 
each  time  the  models  were  linked  after  software  changes. 

Testing  for  the  initial  1992  ALSP  Confederation  consisted 
of  an  integration  test,  two  functional  validations,  and 
stabilization  testing  prior  to  each  exercise.  The  goal  was  to 
freeze  software  development  far  enough  ahead  of  each  exercise 
so  that  it  could  be  stabilized  by  testing  prior  to  start  of 
exercise  (STARTEX) .  Because  of  the  research  nature  of  the 
1992  ALSP  Confederation,  things  did  not  work  that  way. 
Changes  were  still  made  right  up  to  STARTEX,  and  the 


^Seidel,  David  W..  The  Aggregate  Level  Simulation  Protocol  Program  Status  and  History.  MITRE 
Corporation,  McLean,  VA  1993 
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predictable  problems  associated  with  these  last  minute  changes 
occurred.  Many  occurred  haphazardly  and  without  configuration 
control . 

Another  problem  area  was  reconciliation  of  databases. 
Since  weapon  and  equipment  types  often  do  not  match  between 
simulations,  a  mapping  function  is  used  to  handle  differences. 
When  data  is  sent  from  one  simulation  to  another,  all  data 
values  are  converted  first  from  originator  simulation  data 
values  to  ALSP  values,  and  then  from  ALSP  values  to  receiver 
data  values .  Unfortunately,  the  precise  meaning  of  a  datum 
often  depends  on  its  specific  usage  within  each  simulation. 
This  problem  was  obvious,  but  its  difficulty  did  not  become 
clear  until  after  the  exercises  had  begun.  As  an  example.  Air 
Force  (AF)  players  had  a  difficult  time  bombing  CBS  targets 
because  CBS  attrition  algorithms  were  being  used. 

A  problem  area  is  that  the  Confederation  requires  more 
computer  resources  than  a  single  simulation  because  it  is 
composed  of  multiple  simulations  communicating  via  a  common 
protocol.  As  such  the  Confederation  is  not  an  entity  with 
built-in- test  capabilities.  While  component  simulations 
commonly  have  built-in- test  facilities,  but  these  do  not 
support  a  confederated  operation.  This  makes  the  evaluation 
of  test  results  more  difficult.  Serious  problems  were 
encountered  during  exercises  when  one  simulation  sent  messages 
containing  values  not  expected  by  another.  In  one  case,  the 
messages  contained  invalid  ALSP  identifiers.  This  caused 
major  data  corruption  in  the  receiving  simulation.  The 
simulations  generally  do  not  exhibit  robustness  when 
confronted  with  bad  data  inputs. 

Performance  testing,  even  for  stand  alone  simulations,  is 
difficult,  and  it  is  extremely  difficult  for  the 
Confederation.  Since  the  performance  of  the  Confederation  is 
very  sensitive  to  workload,  it  is  essential  that  valid 
characteristic  workloads  be  used  in  performance  testing. 
Since  each  exercise  has  unique  performance  characteristics  it 
is  essential  that  exercise  planners  provide  realistic  load 
parameters  to  be  used  during  testing.  Return  of  Forces  to 
Germany  (REFORGER)  92  provides  a  good  example  of  how  these 
unique  performance  characteristics  can  effect  the 
Confederation  performance.  It  was  a  two  part  exercise.  One 
part  was  a  BCTP  Warfighter  Exercise.  The  other  part  was 
REFORGER  92  proper.  The  performance  of  the  Confederation  was 
vastly  different  between' these  two  parts.  The  Warfighter  was 
a  fast  paced  exercise  with  heavy  combat  and  large  numbers  of 
air  missions.  At  several  points  it  was  in  danger  of  failure 
due  to  loss  of  clock  time  performance.  On  the  other  hand, 
REFORGER  92  proper,  being  slower  paced,  experienced  few 
performance  problems. 

As  the  confederation  grows,  testing  complexity  will 
increase.  Each  time  a  new  simulation  is  added  to  the 
confederation,  every  current  member  simulation  must  be  tested 
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against  the  new  participant.  Even  though  two  simulations  may- 
have  no  direct  interface,  the  presence  of  their  message 
traffic  may  have  unintended  effects  on  each  other.  For 
example  during  UFL  92,  RESA  was  added  to  the  Confederation. 
It  was  intended  that  RESA  only  participate  in  the  time 
synchronization  aspects  of  ALSP.  Since  its  ALSP  interface  was 
not  adequately  tested,  it  was  to  use  its  existing  direct 
interface  with  AWSIM.  All  of  the  new  code  that  interfaced 
with  AWSIM  via  ALSP  was  supposedly  commented  out.  However, 
some  code  was  apparently  missed  and  RESA  sent  several 
erroneous  messages  to  CBS  that  could  have  had  disastrous 
consequences . 

In  1992  following  the  successful  use  of  the  research 
Confederation,  Defense  Advanced  Research  Projects  Agency 
(DARPA)  determined  the  ALSP  project  was  now  a  functional 
project  and  needed  to  transition  to  the  Services  for 
management  of  future  developments.  A  request  for  an 
organization  to  accept  management  responsibility  was  refused 
by  the  Joint  Staff  and  other  Services,  so  the  Army  was  tasked 
to  take  responsibility  for  future  ALSP  Confederations .  A 
Memorandum  of  Agreement  between  DARPA  and  the  USA  Simulation, 
Training  and  Instr-umentation  Command  (STRICOM)  formalized  the 
transition  of  responsibility  for  ALSP  to  the  Army. 

In  March  1993,  the  DoD  Executive  Council  for  Modeling  and 
Simulations  approved  the  ALSP  Management  Plan  and  formally 
designated  STRICOM  as  the  Executive  Agent  for  ALSP.  This 
provided  the  means  for  bringing  a  focus  and  structure  to  the 
ALSP  Confederation  Development  and  Testing  processes.  The 
identification  of  the  DoD  focal  point  for  ALSP  has  made  it 
possible  to  begin  to  alleviate  many  of  the  concerns  and  issues 
raised  during  the  1992  use  of  the  research  Confederation. 

1.2  Management  Plan 

The  management  of  the  1994  ALSP  CW&A  process  requires 
monitoring  and  coordination  of  tasks  both  within  (internal  to) 
and  among  the  agencies  involved.  This  section  will  describe 
and  discuss  the  methodology  and  task  structure  for  development 
of  a  CWAMP  and  for  carrying  out  the  resultant  W&A 
procedures . 

1.2.1  Management  Philosophy 

The  W&A  process  for  the  1994  ALSP  Confederation  will 
focus  on  two  objectives: 

a.  To  ensure  timely  and  effective  coordination  of  the 
relevant  activities  of  all  members  of  the  ALSP  community. 

b.  To  initiate  configuration  control  policies  for  the 
1994  ALSP  Confederation  of  Models. 
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Conducting  W&A  procedures  and  reporting  the  results  may 
have  profound  organizational  impacts,  and  these  impacts  raise 
concerns  in  the  organizations  effected.  The  most  effective 
way  to  address  these  concerns  is  to  adopt  a  team  structured 
approach.  This  allows  concerns  to  be  raised  and  addressed 
directly  in  a  non-adversarial  setting,  and  leads  to  a  spirit 
of  commitment  and  cooperation. 

In  the  special  case  where  the  W&A  Sponsor  and  Developer 
are  related,  special  efforts  must  be  made  to  preserve  the 
objectivity  and  accuracy  of  negative  assessments  against 
normal  organizational  pressures  to  minimize  bad  news.  A  team 
approach  emphasizes  group  commitments  and  responsibilities, 
rather  than  identifying  culprits  for  problems,  and  so  reduces 
apprehension  and  increases  acceptance  of  W&A  results,  whether 
they  are  'good'  or  'bad'.  Also,  because  of  their  complexity, 
ALSP  CWA  efforts  are  inherently  team  undertakings.  This  fact 
underlies  the  methodology . of  CWA  planning.  The  composition 
of  a  W&A  team  depends  on  the  sponsor  of  the  W&A  effort,  the 
simulation  developer  and  proponent,  the  relationship  between 
them,  the  type  of  model,  and  the  resources  available. 
Naturally,  limitations  on  resources  may  restrict  the  scope  of 
a  W&A  effort. 

The  important  players  in  a  W&A  effort  include  the 
following : 

a.  Management.  W&A  efforts  are  most  effective  when, 
once  a  team  is  selected  and  instructed  as  to  the  means 
available  them  and  their  overall  purpose,  they  are  then 
allowed  to  proceed  without  further  interference  to  the 
conclusion  their  efforts.  Management  has  the  difficult  task 
of  maintaining  oversight  without  interfering  in  the 
assessment,  so  long  as  it  proceeds  within  the  parameters 
established  at  the  outset.  Management  is  also  responsible  for 
the  coordination  of  efforts  among  organizations  involved  in 
the  assessment  effort. 

b.  Developer.  Obtaining  the  confidence  and  cooperation 
of  the  development  team  greatly  facilitates  an  assessment 
effort.  Both  the  original  implementers  of  the  system  (when 
available)  and  the  current  staff  are  important.  Implementers 
can  provide  essential  insights  into  the  design  of  the  system, 
permitting  a  more  sophisticated  assessment.  Current  staff  are 
often  essential,  and  always  desirable,  in  the  operation  of  the 
system  during  the  W&A  procedures . 

c.  User  Community.  It  is  frequently  the  user  community 
of  a  simulation  tool  that  sponsors  an  assessment.  Their 
common  intent  is  to  determine  the  credibility  and  reliability 
of  the  simulation  in  meeting  their  specific  needs.  Potential 
or  actual  users  of  a  simulation  play  an  important  role  in 
identifying  the  goals  of  an  assessment.  Specific  application 
of  the  simulation  helps  focus  analysis  and  assessment  on  the 
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critical  areas  of  capability  and  risk  that  characterize  the 
model . 

d.  Assessment  Team.  The  ALSP  Confederation  Assessment 
Team  is  composed  of  the  model  developers  and  systems 
engineers.  They  will  provide  the  specialized  expertise 
required  to  determine  the  validity  of  the  model  as  a 
representation  of  reality.  These  experts  may  come  from  any 
source,  provided  they  have  the  necessary  expertise.  Technical 
experts  are  acquainted  with  the  host  hardware  and  software 
architectures,  software  engineering  methodologies  employed  in 
development,  and  the  needs  and  abilities  of  both  the  technical 
and  user  communities  for  the  model. 

e.  Accreditation  Team.  The  Accreditation  Team  will 
conduct  timely  reviews  of  the  data  gathered  during 
confederation  testing  and  produce  the  accreditation  document 
for  the  ALSP  Community.  Since  the  Confederation  spans 
multiple  military  organizations,  it  is  important  that  each 
organization  be  represented  on  the  accreditation  team.  The 
ALSP  Review  Panel  will  review  the  final  test  results  and  make 
decisions  relative  to  accreditation  of  the  ALSP  Confederation. 

1.3  Information  Resources 

Information  from  many  sources  will  be  collected  to 
successfully  complete  the  1994  ALSP  CWA  process.  This 
information  will  be  used  to  detail  the  V&V  process,  to  develop 
test  parameters  and  procedures,  and  to  support  W&A 
conclusions.  Some  of  the  information  resources  are  described 
in  the  following  sections . 

1.3.1  M&S  Documentation 

Documents  referencing  the  M&S  of  the  confederation  will 
be  used  to  determine  the  stand  alone  functions  of  each  M&S  of 
the  confederation.  The  operation  and  results  of  these  stand 
alone  functions  will  be  compared  with  the  operation  and 
results  of  the  confederation  to  determine  the  effect  of  the 
confederation  on  each  model .  Many  models  in  the  confederation 
already  conform  to  the  standard  for  documentation  outline  in 
Department  of  Army  Pamphlet  5-11.  The  documentation  available 
is  classified  as  follows : 

Executive  Overview  -  Provides  a  broad  overview  of  the 
model.  It  typically  describes  the  general  characteristics  of 
the  model,  and  provides  enough  administrative  and  technical 
information  to  assist  a  potential  M&S  user  or  application 
sponsor  in  the  selection  of  an  M&S  for  a  particular 
application. 

User's  Manual  -  Documents  each  component  of  an  M&S,  and 
provides  assistance  to  the  actual  users  in  the  proper 
operation  and  application  of  the  model. 
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Installation  Guide  -  Set  of  detailed  instructions  that 
walk  the  user  through  the  installation  of  all  M&S  components . 
It  typically  includes  such  items  as  required  hardware  and 
software,  and  environmental  specifications. 

Operator's  Manual  -  Provides  the  user  with  the  detailed 
knowledge  necessary  to  properly  operate  the  model.  This 
includes  initialization  procedures,  default  settings,  and 
recovery  actions. 

Data  Dictionary  -  Provides  the  user  with  a  listing  of  all 
input  data  items,  such  as  system  component  characteristics, 
force  structure,  and  geographical  data. 

Analyst  Guide  -  Documents  detailed  and  technical 
descriptions  of  the  model's  algorithms  and  methodologies. 

1.3.2  M&S  Developers 

The  developers  of  each  M&S  should  prove  invaluable  in 
determining  the  methodologies  and  operating  assumptions  of 
their  respective  models.  The  CWAMP  should  identify  Point  of 
Contacts  (POCs)  for  each  of  the  models  and  developer  Subject 
Matter  Experts  (SMEs)  that  can  be  contacted  to  evaluate  issues 
that  may  surface  during  the  W&A  process . 

1.3.3  SMEs 

SMEs  will  be  used  in  the  V&V  process.  The  SMEs  will 
perform  the  functional  tests  in  each  functional  area  of  the 
individual  models  in  the  confederation.  They  will  also  record 
the  results  of  these  tests  and  will  determine  the  validity  of 
the  results. 

1.3.4  ICDs 

The  ICDs  serve  as  the  foundation  of  the  confederation's 
verification  requirements.  The  ICDs  document  the  operational 
requirements  of  the  interface,  as  determined  by  the  ALSP 
Interface  Working  Group.  They  also  describe  modeling  and 
implementation  concepts.  The  Confederation  V&V  will  include 
tests  to  determine  whether  or  not  the  ALSP  interface  operates 
in  accordance  with  these  concepts,  and  whether  the 
implementation  of  these  concepts  meets  the  operational 
requirements . 

1.3.5  ICD  Test  Plans 

The  ICD  test  plans  are  used  by  model  developers, 
proponents,  and  application  sponsors  to  test  each  ALSP 
interface,  determining  whether  or  not  the  ALSP  interface 
operates  in  accordance  with  the  ICD's  operational  concepts, 
and  whether  the  implementation  of  these  concepts  meets  the 
operational  requirements  as  published  in  the  ICDs . 
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1.4  Technical  Testing 

Before  progressing  to  functional  issues,  testing  must  be 
performed  to  ensure  that  the  ALSP  Confederation  is  technically 
sound  and  that  procedures  designed  to  keep  models  operational 
and  synchronized  work  properly. 

1 . 4 . 1  Technical  Test  Objectives 

Technical  items  to  be  tested  include  all  functions 
necessary  to  start,  stop  and  maintain  the  simulations,  both  in 
stand  alone  mode  and  as  a'  member  of  the  ALSP  Confederation. 
Such  items  include,  but  are  not  limited  to,  the  following: 

Start:  Does  each  model  correct  start  up  and  properly  read 
in  all  necessary  data  files? 

Join  and  Resign:  Does  each  model  correctly  transition 
between  linked  and  unlinked  modes  of  operation? 

Time  synchronization:  Is  causality  across  the 

confederation  maintained? 

Checkpoint  and  Restart:  Can  the  entire  confederation  take 
a  joint  checkpoint  and  restart  from  that  checkpoint 
following  the  published  checkpoint  and  restart 
procedures? 

1.4.2  Technical  Tests 

Technical  testing  should  be  performed  as  part  of  each 
test  involving  the  ALSP  Confederation.  These  types  of  tests 
include : 

a.  Any  verification  tests  that  are  a  part  of  the 
development  cycle  for  each  respective  model. 

b.  Stand  alone  tests  to  ensure  that  each  model  is 
"confederation  ready" . 

c.  Integration  tests  (ITs)  among  collections  of  two  or 
more  component  models,  to  be  run  by  developers  prior  to 
Confederation  testing.  Examples  are  Jet  Propulsion  Laboratory 
(JPL)  and  Combined  Arms  Support  Command  (CASCOM)  efforts  with 
CBS  and  CSSTSS,  and  the  Developers'  IT  which  took  place  at  the 
NSC  from  1-18  February  1994. 

d.  Confederation  Tests  (CTs) .  Technical  testing  is  one 
of  the  primary  goals  of  the  Main  CT. 

1.5  Verification  Testing 

Because  the  Confederation  consists  of  a  group  of 
individual  models  communicating  through  ALSP,  it  needs  to  be 
verified  by  means  of  two  series  of  tests:  stand  alone  tests 
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of  each  the  component  model  systems,  and  interface 
verification  tests  between  collections  of  two  or  more  of  the 
component  systems. 

1.5.1  Verification  Test  Objectives 

The  items  that  will  be  verified  include:  the  component 
models,  their  ALSP  translators,  the  ALSP  System  Software,  and 
the  1994  ALSP  Confederation.  Verification  should  be  designed 
to  answer  the  following  questions: 

ALSP  Compliance:  Does  each  of  the  translators  correctly 
implement  the  simulation  management  protocols  specified 
in  the  ALSP  Technical  Specification? 

Functional  Interface:  Does  each  of  the  model- to-model 
interfaces  (translator  pairs)  correctly  implement  the 
functional  interface  protocol  specified  by  the 
corresponding  ICDs? 

1.5.2  Verification  Tests 

Any  of  the  following  types  of  tests  can  serve  as  part  of 
the  verification  testing  for  the  ALSP  Confederation: 

Component  tests  that  are  a  part  of  the  development  cycle 
for  each  participating  model.  These  are  to  specifically 
include  any  design  reviews,  code  walk  throughs,  unit  tests, 
and  other  formal  testing  procedures.  This  category  of  tests 
is  the  responsibility  of  the  model  developer  and  model 
proponent . 

Functional  interface  tests  that  ensure  each  model  is 
"confederation  ready" .  These  tests  are  a  series  of  tests 
among  the  models  sharing  a  functional'  interface  and  are  the 
responsibility  of  the  model  developers  and  model  proponents . 

ALSP  Confederation  IT  is  the  developers  test  involving 
all  models  participating  in  the  ALSP  Confederation.  Models 
participating  in  this  test  affirms  that  all  components  and 
functional  interface  testing  has  been  completed. 

ALSP  CT  is  the  users  test  involving  all  models 
participating  in  the  ALSP  Confederation  and  others  models 
providing  non-ALSP  point-to-point  interfaces  with  models  in 
the  ALSP  Confederation.  This  test  includes  technical, 
functional,  and  load  testing. 

1.6  Validation 

Validation  is  the  rigorous  and  structured  process  of 
determining  the  extent  to  which  a  simulation  accurately 
represents  the  intended  "real  world"  phenomena  from  the 
perspective  of  the  intended  use  of  the  simulation. 
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Interface  validation  testing  can  be  conducted  at 
differing  levels  of  rigor,  depending  of  the  resources 
available: 

a.  Limited  Interface  Validation  Test  (IVT)  -  This  test 
is  executed  by  applying  realistic  parameters  to  the  IVT 
specifications,  and  then  comparing  the  results  with  expected 
values  provided  by  SME. 

b.  Full  IVT  -  Full  IVT  requires  running  test  cases  over 
a  wide  range  of  interface  parameters  and  reviewing  each 
functional  area  of  the  translator  for  each  model.  Test  cases 
must  be  constructed  by  SME's  for  each  functional  area  to  be 
tested.  The  results  from  these  test  cases  are  then  collected 
and  analyzed  by  the  SME  and  compared  with  accepted  historical 
norms . 

Where  the  individual  models  have  already  been  validated, 
it  may  be  sufficient  to  perform  interface  validation.  IVT 
validates  only  those  functional  areas  that  are  directly 
affected  by  the  interface. 

1.6.1  Validation  Objectives 

In  M&S  validation,  the  object  is  to  qualify  the  model  as 
an  accurate  representation  of  reality  for  its  intended  usage. 
With  respect  to  the  ALSP  Confederation  the  object  is  somewhat 
broader:  the  scope  of  the  validation  includes  not  only  the 
individual  models,  but  also  the  operation  of  the  models  when 
joined  as  a  confederation.  Specifically,  the  question  that 
the  validation  of  the  ALSP  Confederation  should  address  is  as 
follows: 

Does  the  confederation  accurately  represent  reality  from 

the  perspective  of  its  intended  use? 

1.6.2  Validation  Tests 

The  following  tests  serve  as  part  of  the  validation  of 
the  ALSP  Confederation: 

a.  Validation  testing  that  is  part  of  the  development 
cycle  for  each  respective  model.  These  are  to  specifically 
include  any  design  reviews,  code  walk  throughs,  unit  tests, 
and  other  formal  testing  procedures.  Each  individual  model 
should  be  validated  by  the  model's  respective  proponent.  The 
results  of  this  validation  will  be  forwarded  to  the 
accreditation  team  and  incorporated  into  the  Confederation 
accreditation  documentation. 

b.  Validation  testing  of  the  entire  Confederation.  This 
is  one  of  the  primary  goals  of  the  CT.  The  results  of  the 
validation  will  be  forwarded  to  the  accreditation  team  along 
with  the  other  results  from  the  CT  for  their  use  in  the 
accreditation  process. 
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1.7  Load  Testing 

A  Load  Test  Plan  will  be  used  to  assess  the 
Confederation's  ability  to  meet  performance  requirements. 
Also,  the  sensitivity  of  the  Confederation's  performance 
(speed)  to  certain  exercise  parameters  will  be  tested.  The 
following  are  some  examples  of  load  test  metrics: 

a.  Effective  game  ratio  -  the  maximiam  ratio  of  game  time 
to  clock  time, 

b.  The  percentage  time  each  model  waits  on  the 
Confederation, 

c.  The  percentage  time  each  model  spends  computing, 

d.  Network  utilization  metrics, 

e.  The  amount  of  time  required  for  Confederation 
checkpoints , 

f.  The  amount  of  time  required  to  refresh  work  stations. 

During  load  testing  internal  monitoring  tools  such  as 
IMON  will  be  used  to  determine  those  subroutines  which  use  the 
highest  amount  of  CPU  percentage.  Inputs  and  Outputs,  etc. 

1.7.1  Load  Test  Objectives 

The  purpose  of  the  Performance  Evaluation  Plan  is  to 
assess  the  overall  performance  of  the  ALSP  Confederation  and 
ensure  that  it  can  perform  adequately  to  meet  joint  training 
exercise  objectives.  The  specific  issues  which  must  be 
addressed  are  as  follows : 

a.  Can  the  ALSP  Confederation  maintain  a  ratio  of, 1:1? 
That  is,  can  game  time  keep  up  with  real  time? 

b.  What  specific  routines,  functions,  entities,  etc. 
consume  the  largest  amount  of  CPU  resources? 

c.  Can  the  local  and  wide  area  networks  used  for  exercise 
support  handle  the  amount  of  data  generated  by  the 
Confederation? 

The  achievement  of  these  objectives  will  allow  exercise 
support  personnel  to  develop  performance  scaling  factors  to 
apply  in  developing  future  performance  estimates  and  planning 
future  exercises  and  determine  possible  methods  of  dealing 
with  future  performance  problems  during  an  exercise. 

1.7.2  Load  Tests 

The  Confederation  Load  Test  is  one  of  the  primary  goals 
of  the  CT.  The  results  of  the  load  test  will  be  analyzed  and 
then  forwarded  to  the  accreditation  team  for  their  use  in  the 
accreditation  process.  The  analysis  of  the  load  test  will 
provide  an  assessment  of  the  abilities  of  the  models  to  meet 
performance  requirements . 
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1.8  Accreditation 


The  accreditation  of  the  ALSP  Confederation  will  be 
performed  by  the  Accreditation  Team.  The  specific 

accreditation  of  the  ALSP  Confederation  for  use  in  an  exercise 
should  be  conducted  by  those  personnel  responsible  for  the 
support  of  that  exercise. 

1.8.1  Accreditation  Methodology 

The  Accreditation  Team  will  be  given  access  to  all 
documentation  generated  and  collected  during  Confederation 
testing.  This  documentation  will  serve  as  the  basis  for  the 
accreditation  decision.  Models  are  given  accreditation  not 
only  to  serve  in  the  targeted  exercises,  but  also  as  a 
memorandum  of  agreement  among  all  affected  parties  as  to  the 
uses  and  limitations  of  each  member  of  the  Confederation.  The 
Accreditation  Team  must  identify  and  acknowledge  problem  areas 
in  order  to  promote  the  most  effective  use  of  ALSP 
Confederation . 

Results  of  confederation  testing  will  be  forwarded  to  the 
Directors  of  Exercise  Support  for  all  major  exercises  planned 
for  the  year.  Each  Director,  if  they  so  choose,  may  send  an 
accreditation  report  and  recommendations  to  the  Exercise 
Director.  The  Exercise  Director,  based  upon  this  information 
and  with  full  awareness  of  the  capabilities  and  limitations  of 
the  ALSP  Confederation,  can  then  make  an  informed  decision 
concerning  accreditation  of  the  ALSP  Confederation  for  use  in 
their  exercise. 

1.8.2  Acceptance  Criteria 

The  following  are  the  criteria  on  which  accreditation 
decisions  for  the  Confederation  of  Models  should  be  based: 

a.  Was  the  Confederation  V&V  thorough? 

b.  Will  the  Confederation  support  its  intended  uses? 

c.  Are  the  risks  associated  with  using  the  Confederation 
for  its  intended  purposes  significant? 

d.  Should  the  intended  uses  of  the  Confederation  as  a 
whole  be  modified  or  constrained  to  mitigate  risks? 
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2 . 1  General 


This  part  comprises  the  CWAMP  for  the  1994  ALSP 
Confederation  of  Models.  The  CWAMP  was  developed  by  applying 
Part  I  of  this  document  to  the  testing  of  the  Confederation  of 
Models .  While  Part  I  describes  a  method  of  applying  a 
rigorous  and  systematic  approach  to  W&A  to  the  ALSP 
Confederation,  Part  II  describes  the  methods  which  were 
actually  used  to  test  and  accredit  1994  ALSP  Confederation, 
given  real  world  constraints  in  time  and  resources. 

2.2  1994  ALSP  Confederation  Management  Plan 

The  management  of  the  1994  ALSP  CWA  process  required 
monitoring  and  coordination  of  tasks  among  the  agencies 
involved.  This  section  describes  the  methodology  and  task 
structure  for  development  of  the  1994  ALSP  CWAMP  and  for 
carrying  out  the  resultant  W&A  procedures . 

2.2.1  Identification  of  Responsible  Agencies  with  POCs 

Appendix  A  provides  a  complete  listing  of  all  agencies 
involved  in  the  1994  ALSP  CWA  process  and  of  POCs  for  each 
agency.  In  addition,  exercise  directors  who  will  be  asked  to 
serve  on  the  Accreditation  Team  are  identified. 

This  section  publishes  and  clarifies  W&A  related 
responsibilities  agreed  to  by  the  specified  agencies  with 
regard  to  ALSP  Confederation.  It  also  serves  to  identify  the 
model  proponents  and  other  interested  agencies. 

a.  ALSP  Executive  Agent  and  Test  Director  -  STRICOM  as 
executive  agent  will  serve  as  the  Test  Director  of  the  W&A 
IT  and  CT.  As  the  Test  Director,  STRICOM  develops  the  test 
plan  for  the  IT. 

b.  ALSP  W&A  Functional  Director  -  NSC  serves  as  the 
Functional  Director  at  the  W&A  CT.  The  responsibilities 
assumed  by  the  NSC  are  to  write  the  CWAMP  and  to  develop  the 
Integrated  Test  Plan  for  the  CT. 

c.  ALSP  W&A  Technical  Director  -  The  MITRE  Corporation 
serve  as  the  Technical  Director  for  W&A  of  1994  ALSP 
Confederation  to  produce  and  maintain  the  ALSP  Confederation 
Master  Calendar.  As  the  Technical  Director,  MITRE  runs  the  CT 
per  the  guidance  of  the  Executive  Agent  and  Test  Director. 

Verification  of  Component  Models  -  The  proponent  agency 
for  each  model  in  the  ALSP  Confederation  is  responsible  for 
V&V  of  their  model.  Models  participating  in  the  1994  ALSP 
Confederation  will  have  been  individually  verified  and 
validated  prior  to  the  ALSP  Confederation  IT. 

Validation  Parameter  Development  -  The  model  proponents 
are  responsible  for  development  of  parameters  to  be  used  in 
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validating  the  Confederation.  Because  of  the  sensitive  nature 
of  this  issue  and  the  inability  to  reach  agreement  among  the 
various  agencies  on  the  proper  methods  and  data  to  be  used, 
validation  of  the  Confederation  was  not  undertaken  for  1994. 

Collection  of  Test  Data  -  The  SMEs  conducting  each 
individual  test  had  the  responsibility  for  recording  test 
results  and  observations.  The  Joint  &  Combined  Division,  M&S 
Directorate,  NSC,  is  responsible  for  the  collection, 
collation,  and  preparation  of  result  functional  data  for 
presentation  to  the  Accreditation  Team.  The  system  engineer 
collected  technical  and  load  data.  They  are  responsible  for 
collecting  all  data  and  presenting  it  to  the  ALSP  Review 
Panel . 

Accreditation  -  The  Accreditation  Team  is  comprised  of 
the  ALSP  Review  Panel .  They  are  responsible  for  preparation 
of  the  Accreditation  Recommendation  Report  for  ALSP  CT  and  to 
provide  the  report  to  the  Application  Sponsors  and  the  ALSP 
Executive  Agent.  The  Application  Sponsors  are  the  Exercise 
Directors  for  the  major  exercises  in  which  ALSP  Confederation 
will  be  used.  In  addition,  the  Director  of  the  NSC  prepared 
an  Accreditation  Report  for  the  use  of  the  Augmented 
Confederation  (consisting  of  1994  ALSP  Confederation  and 
CSSTSS)  in  support  of  General  HQ  (GHQ)  Phase  III  94. 

2.2.2  1994  ALSP  Confederation  Management  Methodology 

2. 2. 2.1  WScA  Requirements 

The  models  tested  during  the  1994  CT  are  CBS,  AWSIM, 
RESA,  JECEWSI,  MTWS  and  TACSIM.  The  interfaces  were  tested 
and  the  models  involved  in  each  interface  are  listed  in  Table 
2-1. 


ALSP  Interface 

Models  Involved 

Air-Ground 

CBS,  AWSIM,  RESA,  MTWS 

Air-Air 

AWSIM,  RESA,  MTWS 

Maritime 

AWSIM,  RESA,  CBS 

TBM/CM 

AWSIM,  RESA,  CBS 

Ground  Unit  Initialization 

JECEWSI,  CBS 

TACSIM  Interface 

TACSIM,  CBS 

Sustainment 

CBS,  CSSTSS 

Table  2-1  ALSP  Interfaces  and  Involved  Models 


Test  hosts  were  responsible  for  ensuring  that  the 
operation  requirements  for  each  model  were  fulfilled.  The  NSC 
hosted  the  Developer's  IT  from  1-18  February  1994.  The  WPC 
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hosted  the  1994  CT  from  14  -  25  March  1994.  Model  proponents 
were  responsible  for  delivering  the  test  hosts  their 
requirements  in  the  areas  of  hardware  and  software  support, 
personnel  requirements,  security  procedures,  communications 
systems,  and  operational  parameters. 

2. 2. 2. 2  Development  of  the  Draft  CWAMP 

The  first  draft  of  the  CWAMP  was  begun  at  the  NSC  on 
October  1,  1993.  It  focused  on  the  concepts  and  methods  of 
W&A  of  the  Confederation. 

2. 2. 2. 3  Collection  of  ICD  Test  Plans 


ICD  test  plans  were  written  by  model  developers  and/or 
proponents  for  use  in  the  Developer's  IT.  The  agencies  which 
wrote  the  tests  for  each  ICD  are  as  follows: 


Air-Ground 

Air-Air 

Maritime 

JECEWSI  Ground  Unit 
TACSIM  Interface 
Sustainment 


JPL 

Sonalysts,  Inc. 
Sonalysts ,  Inc . 

CACI  Products  Co. 

The  MITRE  Corporation 
JPL 


2. 2. 2. 4  Deliver  Draft  CWAMP 


A  rough  draft  of  the  CWAMP  was  delivered  by  the  NSC  on 
October  19,  1993.  This  rough  draft  was  revised  and  refined 
based  on  comments  from  the  ALSP  community  and  the  first  draft 
of  the  CWAMP  was  delivered  on  November  8,  1993,  for  the 
review  by  all  involved  and  interested  parties  including  model 
proponents,  model  developers,  SMEs  and  users  of  the 
simulations.  The  draft  CWAMP  will  then  entered  a  comment  and 
review  cycle  among  those  parties.  These  comments  were 
collected,  reviewed,  and  submitted  for  inclusion  in  the  next 
draft  by  the  Configuration  Manager  of  the  CWAMP  (see  section 
2.2.3)  . 


2. 2. 2. 5  Revise  Draft  CWAMP 

Following  extensive  revision  based  on  comments  from  all 
interested  parties,  the  second  draft  of  the  CWAMP  was 
delivered  on  January  14,  1994.  The  first  draft  of  the  ALSP 
Confederation  IT  Plan  was  included  in  this  draft. 

2. 2. 2. 6  Apply  CWAMP  to  WSLA  of  the  Confederation 

Following  another  round  of  review  and  comment,  the  ALSP 
Confederation  IT  Plan  was  used  to  execute  the  functional 
verification  testing  of  the  Confederation  of  Models  during  the 
1994  ALSP  CT  held  at  the  WPC  from  14  -  25  March  1994.  This 
plan  was  executed  and  the  results  were  collected  and  analyzed 
in  accordance  with  the  CWAMP. 
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2. 2. 2. 7  Present  V&V  Data  to  Developers  and  Accreditation 
Tecun 

Data  collected  by  the  assessment  team  during  CT  was 
collated  and  presented  to  the  CT  Director  for  review  and 
comment.  Following  this,  the  accreditation  package  was 
delivered  to  the  ALSP  Review  Panel,  which  served  as  the 
Accreditation  Team  for  the  ALSP  community.  In  addition,  the 
accreditation  report  was  updated  based  upon  the  results  of  GHQ 
94  CT  conducted  from  5-17  May  1994;  and  an  Accreditation 
Report  for  the  use  of  1994  ALSP  Confederation  in  support  of 
GHQ  94  was  prepared  and  sent  to  the  GHQ  94  Exercise  Director 
by  the  Director,  NSC. 

2. 2. 2. 8  Evaluate  CWAMP 

The  CWAMP  should  be  reviewed  and  evaluated  on  the  basis 
of  experience  gained  during  GHQ  94  on  the  basis  of  relevance, 
effectiveness,  and  completeness  as  a  W&A  tool  for  the 
Confederation . 

2. 2. 2. 9  Enhance  CWAMP  for  Future  Confederations 

The  lessons  learned  from  GHQ  94  have  been  incorporated 
into  this  final  version  of  the  CWAMP  for  1994  in  order  to  aid 
in  the  testing  and  accreditation  of  the  ALSP  Confederation  for 
1995. 

2.2.3  Conf Igurat ion  Management 

Development  of  CWAMP  continued  throughout  the  life  cycle 
of  CWA  effort.  Because  of  the  developmental  character  of  the 
plan,  a  configuration  management  discipline  was  applied  to  the 
CWAMP  to  ensure  operational  consistency  and  cooperation  among 
the  agencies  involved. 

The  Chief,  Joint  &  Combined  Division  of  the  NSC,  M&S 
Directorate  was  appointed  Configuration  Manager  for  the 
CWAMP.  As  Configuration  Manager,  he  received  and  dated  the 
initial  draft  of  the  CWAMP,  received  and  collated  comments  on 
the  CWAMP  from  model  developers  and  proponents  and  other 
interested  parties,  and  ensure  the  integration  of  those 
comments  into  the  CWAMP . 

2.2.4  Configuration  Control 

Because  no  one  agency  has  control  over  every  actor  which 
interfaces  to  the  Confederation,  Configuration  Control 
Managers  (CCMs)  for  each  model  and  the  ALSP  software  itself 
were  responsible  for  ensuring  that  those  models  used  in 
Confederation  Exercises  are  the  same  version  as  those  tested 
and  verified  during  the  CT.  Minor  changes  and  bug  fixes 
provided  for  models  following  the  CT  were  thoroughly  tested 
prior  to  use  in  GHQ  94. 
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The  Chief,  Joint  &  Combined  Division,  M&S  Directorate, 
NSC  is  presently  serving  as  the  CCM  for  all  FAMSIM  models 
participating  in  the  Confederation  including  CBS,  CSSTSS,  and 
TACSIM.  As  CCM,  he  also  coordinates  with  the  CCMs  of  the 
other  confederation  models  in  order  to  keep  track  of 
modifications  and  verification  of  those  modifications.  This 
function  will  only  be  in  place  for  1994. 

2.2.5  ALSP  Confederation  Calendar  of  Events 


Appendix  B  contains  the  ALSP  Confederation  Calendar  of 
Events  which  was  used  through  the  verification  life  cycle. 
The  ALSP  W&A  Technical  Director  is  the  POC  for  any  revisions 
to  the  calendar. 


2.3  1994  ALSP  Confederation  Information  Resources 

2.3.1  1994  ALSP  Confederation  M&S  Developers 

Developers  of  the  models  participating  in  1994  ALSP 
Confederation  are  as  follows: 

CBS  JPL 

4800  Oak  Grove  Drive 
Pasadena,  California  91109 

AWSIM  Los  Alamos  National  Laboratories  (LANL) 

Los  Alamos,  New  Mexico  87545 


RESA 


TACSIM 


JECEWSI 


MTWS 


CSSTSS 


Naval  Command,  Control,  and  Ocean 
Surveillance  Center  Research, 
Development,  Test,  and  Evaluation 
Division  (NRaD) 

271  Catalina  Boulevard 
San  Diego,  California  92152 

TACSIM  Project  Office 
7900  Sudley  Road,  Suite  500 
Manassas,  Virginia  22110 

Joint  C2  Warfare  Center 
2  Hall  Boulevard,  Suite  217 
San  Antonio,  Texas  78250 

VisiCom,  Inc. 

100052  Mesa  Ridge  Court 
San  Diego,  California  92121 

CASCOM 

Building  1109 

Fort  Lee,  Virginia  23801 
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2.3.2  ICDs 


CBS-AWSIM  Air-Ground  ICD  and  CBS/CSSTSS  Sustainment  ICD 
Written  By:  JPL 

California  Institute  of  Technology 
Pasadena,  California 

ALSP  Air-Air  Engagement  ICD,  Maritime  ALSP  ICD,  and  ALSP 
Theater  Ballistic  Missile  and  Cruise  Missile  (TBM/CM)  ICD 

Written  By:  Sonalysts,  Inc. 

5675  Ruffin  Road,  Suite  210 
San  Diego,  California 

JECEWSI  ICD  for  Ground  Unit  Initialization 

Written  By:  CACI  Products  Company 
Electronic  Simulation  Department 
San  Antonio,  Texas 

2.3.3  ICD  Test  Plans 

CBS  1.5  Functional  Test  Plan/AWSIM  Interface  and 
CSSTSS/CBS  Interface  Test  Plan 

Written  By:  JPL 

California  Institute  of  Technology 
Pasadena,  California 

ALSP  Test  Plan:  Combat  Interactions  (1994) 

Written  By:  Sonalysts,  Inc. 

5675  Ruffin  Road,  Suite  210 
San  Diego,  California 

JECEWSI  Test  Plan  for  Ground  Unit  Initialization 

Written  By:  CACI  Products  Company 
Electronic  Simulation  Department 
San  Antonio,  Texas 

2.3.4  ALSP  Documentation 

Several  ALSP  documents  are  available  from  the  MITRE 
Corporation.  These  documents  listed  below  provide  a  broad 
range  of  information  on  ALSP  from  a  general  overview  to 
detailed  technical  description  of  the  protocol.  The  documents 
can  be  obtained  via  anonymous  ftp  from  alsp.arpa.mil.  The 
documents  are  the  ALSP  Executive  Overview,  the  ALSP  Technical 
Specification,  the  ALSP  System  Test  Plan,  and  the  ALSP 
Managemen  t  Plan. 
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2.4  ALSP  Confederation  Technical  Testing 

2.4.1  1994  CT 

Technical  testing  was  conducted  as  the  first  phase  of  the 
1994  ALSP  CT  from  0800  March  15  through  1215  March  16.  The 
Technical  Test  was  directed  by  the  Technical  Test  Director 
according  to  the  ALSP  Confederation  Management  Technical  Test 
Plan^.  The  test  was  designed  to  verify  that  actors  in  the 
Confederation  demonstrate  sufficient  technical 
interoperability  to  support  an  exercise.  The  test  also 
verified  the  ALSP  System  Software. 

2.4.2  6HQ  94  CT 

The  technical  testing  of  the  ALSP  Confederation  was 
performed  on  2  -  3  May  1994.  The  test  plan  which  was  executed 
was  the  same  one  conducted  at  the  CT  in  March  1994. 

2.5  Verification  Testing 

2.5.1  1994  CT 

Verification  testing  was  conducted  from  1300  March  16 
through  1215  March  20.  The  Verification  Test  was  directed  by 
the  Functional  Test  Director  according  to  the  CWAMP^ .  The 
test  was  designed  to  verify  and  to  validate,  to  the  extent 
possible,  the  functional  interfaces  between  the  actors  in  the 
Confederation. 

2.5.2  GHQ  94  CT 

Verification  testing  was  performed  on  the  1994  ALSP 
Confederation  as  part  of  the  GHQ  94  pre-exercise  confederation 
testing.  Testing  was  performed  on  CBS  and  CSSTSS  from  5-6 
May  and  on  the  entire  ALSP  Confederation  from  7-13  May.  The 
verification  testing  was  comprised  of  36  separate  tests  from 
the  CWAMP.  Of  these  tests  13  had  either  failed  or  only 
partially  succeeded  at  1994  CT  in  March.  The  remaining  23 
tests  covered  areas  for  which  it  was  felt  that  more  detailed 
test  results  were  required  in  order  to  increase  our  confidence 
in  the  previous  test  results. 


^ALSP  Confederation  Management  Technical  Test  Plan,  The  MITRE  Corporation,  McLean,  VA,  January 

1994. 

^Confederation  Verification,  Validation,  and  Accreditation  Master  Plan  (CWAMP),  Part  III,  Confederation 
’94  Integrated  Test  Plan,  U.  S.  Army  National  Simulation  Center,  Ft.  Leavenworth,  KS,  March  1994.  (Part  HI  of 
the  CWAMP  is  the  integrated  product  of  all  developer  test  plans  for  the  functional  interfaces). 
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2.6  Validation  Testing 

The  services  and  agencies  involved  in  the  ALSP  community 
were  unable  to  come  to  a  consensus  on  the  proper  methods  to 
employ  in  the  validation  testing  and  assessment  processes  for 
the  ALSP  Confederation.  Because  of  this,  validation  testing 
of  the  Confederation  was  not  performed  for  1994. 
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Part  III 


1994  ALSP 
Conf  ederat ion 
Test  Plans 


29 


3 . 1  General 


This  section  provides  a  listing  and  overview  of  the  test 
plans  generated  for  1994  ALSP  Confederation  to  include 
technical,  functional,  and  load  test  plans. 

3.2  ICD  Test  Plans 

The  ICD  Test  Plans  were  written  by  the  model  developers 
in  order  to  test  the  technical  aspects  of  the  ALSP  interfaces. 
These  test  plans  were  used  at  the  Developer's  IT. 

3.2.1  CBS  1.5  ALSP  Functional  Test  Plan  -  AWSIM  Interface 

This  test  plan  written  by  JPL  is  intended  to  test  the 
Confederation's  implementation  of  the  CBS-AWSIM  air-ground 
interface.  Although  the  latest  draft  of  this  ICD  is  titled 
CBS-AWSIM  Interface  Control  Document ,  it  is  in  actuality  the 
ICD  detailing  the  Air-Ground  interface  for  the  Confederation 
(CBS,  AWSIM,  RESA,  and  MTWS) . 

3.2.2  CBS  and  CSSTSS  Interface  Test  Plan 

This  test  plan  written  by  JPL  will  verify  the 
confederation's  sustainment  functionalities  included  in  the 
CBS/CSSTSS  ICD. 

3.2.3  ALSP  Test  Plan:  Coxnbat  Interactions  (1994) 

The  above  titled  test  plan  was  written  by  Sonalysts,  Inc. 
for  NRaD.  This  test  plan  covers  the  functional  areas  making 
up  three  ICDs :  the  Air-Air  Engagement  ICD;  the  Maritime  ICD; 
and  the  TBM/CM  ICD.  As  shown  in  the  crosswalk  below,  this 
test  plan  also  covers  combat  interaction  functionalities 
listed  in  the  CBS-AWSIM  ICD. 

3.2.4  JECEWSI  Test  Plan  for  Ground  Unit  Initialization 

The  Test  Plan  for  Ground  Unit  Initialization  via  ALSP 
connection  was  written  for  the  Joint  C2  Warfare  Center  by  CACI 
Products  Company.  The  plan  will  be  used  to  test  the  ability 
of  JECEWSI  to  join  the  confederation  and  receive  ground  order 
of  battle  information  as  covered  in  the  JECEWSI  ICD  for  Ground 
Unit  Initialization. 

3.3  1994  ALSP  Confederation  Technical  Test  Plan 

This  technical  test  plan  for  1994  ALSP  Confederation  was 
written  by  the  MITRE  Corporation.  The  test  was  designed  to 
verify  that  actors  in  the  Confederation  demonstrate  sufficient 
technical  interoperability  to  support  an  exercise.  The  test 
also  verifies  the  ALSP  System  Software.  This  test  plan  is 
included  in  this  document  as  Enclosure  1. 
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3.4  1994  ALSP  Confederation  Integrated  Test  Plan 

The  IT  Plan  for  1994  ALSP  Confederation  was  use  during 
the  functional  test  phase  of  the  1994  CT.  It  was  designed  to 
ensure  that  all  functionalities  listed  within  the  ICDs 
operated  correctly.  The  test  was  developed  by  compiling  the 
individual  ICD  test  plans  and  modifying  them  to  redirecting 
their  focus  to  the  functional  aspects  of  the  interfaces.  In 
addition,  the  tests  were  fitted  into  a  common  format,  and  test 
steps  and  functionalities  were  broken  into  discrete  steps 
required  by  each  actor.  This  test  plan  is  included  in  this 
document  as  Enclosure  2 . 

3.5  1994  ALSP  Confederation  Load  Test  Plan 

The  1994  ALSP  Confederation  Load  Test  Plan  was  written  by 
the  MITRE  Corporation  using  inputs  provided  by  the  ALSP 
Community  at  the  1994  CT.  Its  purpose  is  to  provide  the  ALSP 
community  with  a  plan  for  testing  the  overall  performance  of 
the  Confederation.  Key  parameters  for  each  model  were 
exercised,  and  measurements  of  system  and  model  performance 
taken  to  determine  the  Confederation's  ability  to  function 
properly  under  the  load  conditions  expected  during 
confederation  exercises  in  1994.  This  plan  is  included  in 
this  document  as  Enclosure  3 . 

3.6  GHQ  94  CT  Plan 

Functional  testing  was  performed  on  ALSP  Confederation  as 
part  of  the  GHQ  94  pre-exercise  confederation  testing.  The 
functional  test  was  comprised  of  36  separate  tests  from  the 
CWAMP .  Of  these  tests  13  had  either  failed  or  only  partially 
succeeded  at  1994  CT  in  March.  The  remaining  23  tests  covered 
areas  for  which  it  was  felt  that  more  detailed  test  results 
were  required  in  order  to  increase  our  confidence  in  the 
previous  test  results.  This  test  plan  is  included  in  this 
document  as  Enclosure  4. 
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Appendix  A  -  1994  ALSP  Confederation  Agencies  and  POCs 


STRICOM 

LTG  Paul  Izzo 

Dr.  Mary  G.  Fischer 

(407)  381-8821 
(407)  381-8836 

MITRE 

Ms.  Anita  Adams 

Mr.  Gordon  Miller 

(703)  883-1389 
(703)  883-5946 

AF  XOMW 

GOL  Michael  Upson 

MAJ  Jack  L.  Jordan 

(202)  504-4441 
(703)  593-5745 

ENWGS 

LGDR  John  Antonelli 

Ms.  Mona  Freilich  (GSC) 

DSN  332-3125 
(609)  234-1100 

WPG 

GOL  Patrick  Webb 

MAJ  Ghris  Shultz 

Mr.  Dave  Perme  (LANL) 

011-49-631-535- 

(505)  655-6610 

-6605 

JECEWSI 

Ms .  Garol  Oakes 

MAJ  James  R.  Robertson 

LT  Kevin  J.  St ever  (USN) 

(310)  977-2579 

(310)  977-4728 

DMSO 

LTG  David  Bartlett 

(703)  998-0660 

RESA 

(NRaD) 

Dr.  Fitzgerald 

Mr.  Ghris  Burns 

(519)  553-3620 

CSSTSS 

(CASCOM) 

Mr.  Thomas  A.  Miller 

Mr.  Joseph  J.  Riley 

(804)  765-1788 

TAG SIM 

MAJ  Gary  Allen 

(703)  830-7607 

GBS/JPL 

MAJ  Tim  Metivier 

Mr.  John  Weidner 

Mr .  Trey  Thorton 

(913)  584-8150 
(818)  397-9953 
(818)  397-9953 

JWFG 

LTG  Ross  Dickinson 

(904)  884-3644 

MTWS 

GPT  Gharles  Peabody 

Mr .  Joe  Nerdad 

(703)  640-3276 

J-8 

GAPT  Tony  Kopacz  USN 

(703)  695-9165 

GEGOM 

Ms.  Jackie  Bogner 

Mr .  Hoi  Ta 

(913)  651-2043 
DSN  552-2043 

LAM 

LTG  Stuart 

(913)  684-4042 
DSN  552-4042 

USAREUR 

MAJ  Steve  Hicks 

011-49-314-475- 

6460/ 

OPNAV/ 

N8 

Mr.  George  Phillips 

(70J)  697-3642 
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Appendix  A  -  1994  ALSP  Confederation  Agencies  and  POCs 

(Continued) 


KOREA 

BSC 

TRAC 

BCTP 

NSC 

DCSINT 

21ST 

TAACOM 


Mr .  Jude  Shea 
Mr .  Wayne  Hansom 

Ms .  Annette  Ratzenberger 

LTC  Nicholson 
Mr.  Sid  Linver 

Dr .  Robert  La  Rocque 
COL  Walter  Sharp 
LTC  R.  Howard  Taylor 


COL  Gary  McMillan 
Ms .  Marilyn  Macklin 

Dr.  Michael  Baranick 


011-822-7915-8020 


(913)  684-2522 

(913)  684-5907 
(913)  651-8311 

(913)  684-8101 
(913)  684-8135 
(913)  684-8614 
DSN  552 

(703)  697-7103 
(703)  692-4644 

DSN  403-8698 
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Appendix  B  -  ALSP  Calendar  of  Events 

Unless  otherwise  noted,  all  documents  to  be  delivered  to 
STRICOM  for  distribution  to  all  participants. 


Date 

Task 

Organization 

Status 

1993 

ASAP 

COMSEC  POCs  to  Mr .  Spencer 
at  NSC 

Actor  Model 
developers 

Complete 

Nov 

12 

Actor  Load  Parameters 

Actor  SW 
developers 

Complete 

Nov 

19 

Integrated  List  of  Actor 

Load  Parameters 

MITRE 

Complete 

Nov 

30 

IT  Host  MOI  published 

NSC 

Complete 

Dec 

1 

Actor  required  enumerations 

Actor  Model 
Proponents 

Complete 

Dec 

1 

Draft  for  ALSP  air-ground 
test  plan 

NSC 

Complete 

Dec 

1 

Draft  for  ALSP  air-air  test 
plan 

NRaD 

Complete 

Dec 

1 

Draft  for  ALSP  TBM/CM  test 
plan 

NRaD 

Complete 

Dec 

1 

Draft  for  ALSP  surface- 
ground  test  plan 

NRaD 

Complete 

Dec 

1 

Draft  for  ALSP  surface-air 
test  plan 

WPC 

Ongoing 

Dec 

9 

IT  IPR  #1  at  NSC 

NSC 

Complete 

Dec 

15 

Integrated  Enumeration  List 

MITRE 

Complete 

Dec 

15 

ALSP  confederation  management 
technical  test  plan  -  v7 . 0 

MITRE 

Complete 

Dec 

15 

Draft  for  ALSP  sustainment 
test  plan 

NSC 

Ongoing 

Dec 

15 

Actor  Load  Parameter 

Estimates 

Users 

Ongoing 

1994 

Jan 

1 

ALSP  air-ground  test  plan 

NSC 

Jan 

1 

ALSP  air-air  test  plan 

NRaD 
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Appendix  B  -  ALSF  Calendar  of  Events 

(continued) 


Date _ Task _ Organization  Status 

Jan  1  ALSP  TBM/CM  test  plan  NRaD 

Jan  1  ALSP  surface-ground  test  NRaD 

plan 

Jan  1  ALSP  surface-air  test  plan  WPC 

Jan  1  Draft  for  ALSP  confederation  MITRE  Complete 

management  technical  test 
plan  (v7.1) 

Jan  1  Load  Test  Data  Collection  MITRE  Complete 

plan 

Jan  1  Actor  save/restore  and  WPC 

recovery  procedures  NRAD 

JEWC 

CASCOM 

VisiCom 

JPL 

Jan  1  Hardware  and  personnel  Everyone 

requirements  to  WPC 

Jan  7  Names  of  personnel  attending  Everyone 

IT  to  Drew  Turinski  at  NSC 


Jan 

7 

IT  Enumerations 

NRaD 

Jan 

7 

IT  Host  MOI  published 

NSC 

Jan 

14 

CT 

Enumerations 

NSC 

Jan 

14 

Visit  requests  to  NSC 

All 

contractor 

personnel 

Jan 

15 

CASCOM/ JPL /NSC  Comms  Link 
Test 

NSC 

Jan 

15 

ALSP  sustainment  test  plan 

NSC 

Jan 

15 

MTWS  test  plan 

MSTP 

Jan 

15 

JECEWSI  test  plan 

JEWC 

Jan 

15 

TACSIM/TAT  test  plan 

TPO 

Jan 

15 

Comms  link  tested 

NSC 

JPL 

CASCOM 
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Appendix  B  -  ALSP  Calendar  of  Events 

(continued) 

Date _ Task _ Organization  Status 


Jan 

15 

Draft  for  integrated  air- 
ground,  air-air,  surface- 
ground,  surface-air, 
sustainment,  and  TBM/CM 
fv  test  plan 

NSC 

Jan 

19 

IT  IPR  #2  at  STRICOM 

NSC 

-20 

Jan 

19 

IT  Plan  distributed 

MITRE 

-20 

Jan 

21 

IT  Enumerations 

JPL 

CASCOM 

Jan 

26 

IT  Phase  I  Set-up 

CASCOM 

-28 

NSC 

MITRE 

Jan 

28 

CT 

NRaD 

Enumerations 

WPC 

Jan 

28 

IT  Enumerations  Finalization 

WPC 

Meeting 

NRaD 

MITRE 

JPL 

CASCOM 

Jan 

31 

IT  Phase  I  Comms  and 

CASCOM 

hardware  checks 

NSC 

MITRE 

Feb 

1 

Database  mapping  for  IT 

MITRE  , 

Feb 

1 

Actor  Load  Test  Plans 

Actor 

Model 

Proponents 

Feb 

1 

ALSP  confederation  management 
technical  test  plan  -  v7 . 1 

MITRE 

Feb 

1 

Integrated  save  and  restore 
and  crash  recovery  procedures 

MITRE 

Feb 

1 

Integrated  schedule 

WPC 

Feb 

1 

Start  Phase  I  of  IT 

STRICOM 

JPL 

MITRE 

CASCOM 
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Appendix  B  -  ALSP  Calendar  of  Events 

(continued) 


Date  Task _ Organization 


Feb 

4 

IT  Phase  II  Set-up 

NSC 

-6 

NRaD 

WPC/LANL 

MITRE 

VisiCom 

CACI 

Feb 

7 

IT  Phase  II  Comms  and 

All  IT 

hardware  checks 

participants 

Feb 

8 

Start  Phase  II  of  IT 

All  IT 
participants 

Feb 

8 

CT  Enumerations 

NSC 

Feb 

11 

CT  Enumerations  Finalization 

NSC 

Meeting 

NRaD 

WPC 

MITRE 

CASCOM 

Feb 

15 

Integrated  air-ground,  air- 
air,  surface-ground, 
surface-air,  sustainment, 
and  TBM/CM  fv  test  plan 

NSC 

Feb 

18 

IT  Complete 

All  IT 
participants 

Mar 

1 

Individual  Actor  Mapping 

NRaD 

for  RESA,  AWSIM,  CSSTSS, 

WPC 

and  CBS 

JPL 

CASCOM 

Mar 

1 

Integrated  Load  Test  Plan 

NSC 

Mar 

1 

Non-Actor  Software  Test 

Non-Actor 

Plans 

Software 

Proponents 

Mar 

14 

Start  CT 

All  CT 
participants 

Mar 

23 

Begin  Sustainment  portion 

All  CT 

of  CT 

participants 

Mar 

25 

CT  Complete 

All  CT 

participants 


B  -  4 


Status 


ABE 

ACM 

Actor 

AF 

ALSP 

AWSIM 

BCTP 

BSC 

C2 

C3 

CASCOM 

CBS 

CCM 

CM 

Confederation 

CT 

CSSTSS 

CPX 

CWA 

CWAMP 


Appendix  C  -  Acronyms  &  Glossary 
ALSP  Broadcast  Emulator. 

ALSP  Common  Module. 

A  Confederation  member  with  its  respective 
ALSP  interface  module. 

Air  Force . 

Aggregate  Level  Simulation  Protocol . 

Air  Warfare  Simulation.  A  simulation 
developed  and  used  by  WPC  and  Blue  Flag. 

Battle  Command  Training  Program. 

Battle  Simulation  Center 

Command  and  Control . 

Command,  Control,  and  Communications. 

Combined  Arms  Support  Command.  A  logistics 
support  organization  located  Fort  Lee, 
Virginia. 

Corps  Battle  Simulation.  A  corps  level 
simulation  (formerly  known  as  JESS)  developed 
by  JPL  for  the  US  Readiness  Command.  Now 
under  US  Army  control . 

Configuration  Control  Manager . 

Cruise  missile. 

The  composite  of  all  simulations  in  an  ALSP 
exercise. 

Confederation  Test. 

Combat  Service  Support  Training  Simulation 
System.  A  rear  area  model  maintained  and 
used  by  CASCOM. 

Command  Post  Exercise.  An  exercise  in  which 
commanders  operate  from  their  command  post 
using  simulated  entities  for  their 
subordinates . 

Confederation  W&A.  W&A  of  the 
Confederation  as  a  composite  entity. 

CWA  Master  Plan. 


DARPA 

DOD 

ENW6S 

FAMSIM 

GHQ 

HQ 

ICD 


lEW 

IT 

IVT 

JECEWSI 

JPL 

JTLS 

JWFC 

IiANL 

M&S 

MTWS 

NATO 

NHaD 


Appendix  C  -  Acronyms  &  Glossary 

(continued) 

Defense  Advanced  Research  Projects  Agency. 

The  original  sponsor  of  ALSP. 

Department  of  Defense. 

Enhanced  Naval  Wargaming  System. 

Army  Family  of  Simulations . 

General  Headquarter. 

•Headquarter . 

Interface  Control  Document.  This  document 
describes  the  operational  requirements 
supporting  each  facet  of  the  interface,  the 
implementation  concept  to  support  each 
requirement,  ALSP  messages  that  will  be 
generated,  and  actions  that  will  be  taken  by 
each  simulation  in  response  to  the  messages . 

Intelligence  and  Electronic  Warfare. 

Integration  Test.  Test  of  the  joint 
operation  of  component  modules  already  tested 
separately .' 

Interface  Validation  Test 

Joint  Electronic  Combat  Electronic  Warfare 
Simulation . 

Jet  Propulsion  Laboratory.  Developer  of  CBS. 

Joint  Theater  Level  Simulation. 

Joint  Warfare  Center.  An  exercise  support 
facility  located  at  Hurlburt  Field,  Florida. 

Los  Alamos  National  Laboratory. 

Models  &  Simulations. 

Marine  Air  Ground  Task  Force  Tactical  Warfare 
Simulation . 

North  Atlantic  Treaty  Organization. 

Naval  Command,  Control  and  Ocean  Surveillance 
Center  Research,  Development,  Test  and 
Evaluation  Division.  Developer  of  RESA. 
Located  in  San  Diego,  California. 
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NSC 

POC 

Proponent 

Agency 

W&A 

REFORGER 

RESA 

SME 

STARTEX 

STRICOM 

TACSIM 

TBM 

UPL 

US 

USA 

USAREUR 

V&V 

W&A 

WPG 


Appendix  C  -  Acronyms  &  Glossary 

(continued) 

National  Simulation  Center.  US  Army  exercise 
support  facility  located  in  Fort  Leavenworth, 
Kansas . 

Point  of  Contact.  Official  agent  of 
communication  for  a  specific  agency  or 
activity. 

An  agency  proposing  to  use  an  M&S  for  a 
particular  purpose,  and  wishing  to  have  it 
for  that  use. 

An  exercise  conducted  by  the  WPG. 

Research,  Evaluation  and  Systems  Analysis.  A 
US  Navy  model  maintained  by  NRaD. 

Subject  Matter  Expert.  Person  with 
specialized  knowledge.  Employed  first  to 
develop  requirements  and  algorithms  for 
models,  and  later  to  validate  them. 

Start  of  exercise. 

US  Army  Simulation,  Training  and 
Instriamentation  Command.  Executive  agent  for 

ALSP. 

A  US  Army  Intelligence  simulation. 

Theater  Ballistic  Missile. 

Ulchi  Focus  Lens.  An  exercise  conducted  by 
the  Korean  BSC. 

United  States. 

US  Army . 

USA  in  Europe . 

Verification  is  the  testing  of  software's 
compliance  with  requirements  and 
specifications;  Validation  is  the  evaluation 
of  its  validity  as  a  model  of  'reality'. 

V&V  and  Accreditation.  Accreditation  is  the 
official  acceptance  of  the  software  for  a 
particular  purpose. 

Warrior  Preparation  Center .  A  wargaming 
facility  located  in  Einsiedlerhof ,  Germany. 
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